On the Convergence Properties of Optimal AdaBoost

In this paper, we establish the convergence of the Optimal AdaBoost classifier under mild conditions. We frame AdaBoost as a dynamical system, and provide sufficient conditions for the existence of an invariant measure. Employing tools from ergodic theory, we show that the margin for every example converges. More generally, we prove that the time average of any function of the weights over the examples converges. If the weak learner satisfies some common conditions, the generalization error is not changing much in the limit. We conjecture that these conditions are satisfied on almost every dataset, and show preliminary empirical evidence in support of that conjecture.